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1. INTRODUCTION 

Mathematics is a way and means of thinking. Meanwhile, mathematics learning process is basically 
a mental process in which students are trained to think (mathematically) and obtain knowledge, skill and 
attitude shaping [1]. Mathematical thinking process is a core process of mathematic activity in order to apply 
general thinking ways [2]. 

The process of learning mathematics is considered as an active process of acquiring knowledge. It 
refers to a process in which teachers take role to help students acquire new knowledge and restructure old 
knowledge and so on, rather than as a process in which students passively adopt certain mathematical content 
and acquire comprehensive knowledge [3]. Considering this active process, surely mathematical thinking 
ability is a dynamic and comprehensive activity including doing mathematics [4]. Furthermore, to think 
mathematically it should engage mathematical power. 

After reviewing some literatures [5-8], mathematical power is defined as student ability to use 
mathematics knowledge in non-routine problem solving, by digging deep into it and estimation, logical 
reasoning (reasoning); communicating mathematical ideas (communication); and connecting the ideas in 
mathematics or toother disciplines (connection) in order to encourage confidence and mathematical 
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disposition [4]. Meanwhile, reasoning, communication and connection are components of 

mathematical power [8]. 

National Council of Teachers of Mathematics (NCTM) in the affirmation of “Vision of 
Mathematical Power for All”, argued that problem-centered learning instructions for all students by 
emphasizing on mathematical power development for all students and mathematical power should become 
the integral parts of curriculum [5]. This is reaffirmed in content standard of Indonesian elementary and high 
school curriculum 2013, stating that learning competence for mathematics is to have a trust on mathematical 
power and mathematics usability, which are formed by learning experiences [9]. 

Many researches about mathematical power ability have been published, generally concluding that 
student mathematical power on aspects of communication, connection and reasoning ability is still very low 
[6], [3], [8]. It is because of inappropriate usage on learning model and on choosing assessment. Students’ 
mathematical power assessment should be in wide scoupe and include all aspects (cognitive, psychomotor, 
and affective). Mathematical power assessment could not be considered as a separated and isolated 
competence assessment [6]. Although one of mathematics knowledge aspects might be more emphasized 
than the other in a certain assessment, it should be clear that mathematical power includes all aspects of 
mathematics knowledge and its integration. 

The latest survey results from OECD PISA in 2015, Indonesia is still in the bottom group that is 
ranked 69 out of 76 countries surveyed [10]. The aspects surveyed by PISA in mathematics and science were 
the abilities of understanding, problem solving, reasoning, and communication. Apparently, the tests 
developed by PISA contains ability aspects of mathematical power. Based on PISA survey's ranking in 2015, 
it could be interpreted that mathematical power ability of Junior High School students in Indonesia is 
still low. 

Referring to some researches conducted by experts and the result of OECD PISA survey, it is 
suggested that test questions created by teachers should be under SOLO Taxonomy guidance based on grade 
level. The questions should be non-routine problem-solving questions for the sake of mathematical power 
development. According to the experts, teaching based on problem solving on one hand could contribute to 
greater student thinking activities, which in turn showed greater activities during the class [11],[12]. On the 
other hand, students’ problem solving ability became a measurement for students’ achievement in 
learning mathematics [12]. 

One of recommended learning approaches to improve mathematical power ability is investigation- 
based learning. By investigation, students could learn actively and explore mathematics tasks, and possibly a 
logical solution might be acquired [13]. In short, a learning activity with investigation approach has improved 
reasoning, communication and connection ability; while these three are the main components of 
mathematical power [8]. 

Reviewing based on cognitive aspects of mathematical power and Structure of the Observed 
Learning Outcomes (SOLO) Taxonomy, many of mathematics test questions composed by teachers 
independently or in mathematics teachers’ forum still focus on routine issues and have not accommodated the 
development and improvement of mathematical power and SOLO Taxonomy characteristics based on grade 
level. Therefore, there has been a contradiction between program target and evaluation target. Besides, it is 
clear that assessment is an integral part of learning process as well as a crucial part in improving the 
effectiveness of learning [2]. 

Structure of the Observed Learning Outcomes (SOLO) Taxonomy is a hierarchial model which is 
suitable for measuring subject learning result in different grades for all kinds of tasks [14-16]. SOLO 
Taxonomy classifiesthe ability of students' responses to problems into 5 different hierarchical levels: pre- 
structural, uni-structural, multi-structural, relational and extended abstract. When an assessment is 
conducted in SOLO Taxonomy, the pre-structural level must be excluded from the thinking level, because at 
that stage there is usually no opinion about the topic to learn, or the proposed ideas are irrelevant [15]. 

According to the experts [16], some criteria that could be used to determine whether a test question 
includes to uni-structural, multi-structural, relational or extended abstract are as follows: 

a. A question with Unistructural criteria has all information that could be instantly used to obtain a 
solution. 

b. A question with Multistructural criteria would implicitly need a formula to obtain a solution usingtwo 
or more information and separated from what is in the question. 

c. A question with Relational criteria provides all information but could not be instantly used to obtain a 
solution. The solution is by connecting information provided using general principle or formula to 
acquire new information. Then, based on the new information, solution would be obtained as the final 
answer. 
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d. A question with Extended Abstract criteria provides all information or data, but could not be instantly 
used to acquire final solution. The final solution could be obtained after synthesizing the new 
information. 

Mathematics learning has much greater effort than just helping students to acquire problem solving 
skill and strategy. Furthermore, teacher should attempt to develop positive disposition to learn mathematics — 
aspects which have long-term effects in every things starting from students’ trust on mathematics [17] [18], 
[19], including if mathematics would be their career choice or not [20]. Mathematical power assessment 
should consider student mathematical disposition level in mathematics learning. 

Based on experts’ findings (previous researches), it can be seen that mathematics learning in schools 
has not succeeded significantly in improving mathematical power. It is because the learning model, 
assessment form and assessment instruments have not been properly compiled according to mathematical 
power indicators and SOLO taxonomy characteristic level. Therefore, it is necessary to emerge a new 
formula that is an appropriate assessment model and test and non-test instruments to measure 
mathematical power. 

Model to Assess Mathematical Powerwas fundamentally developed by the authors [6] based on 
portfolio assessment. Author [6] had developed a model to assess mathematical power, by conducting a test 
based on mathematical power indicators. However, the test is in the form of multiple choices, and then it is 
critisized as multiple choice tests which are quite inappropriate and considered not to be able to represent the 
real mathematical power ability of students. Furthermore, in this research, mathematical power test is 
developed in form of essays. On the other hand, NCTM document (1989) suggested a shift in assessment 
practice that is to assess students’ mathematical power entirely; not only from the facts of isolated knowledge 
and skills [4, 21]. 

Assessment model developed in this research is a means to collect data in order to assess or measure 
mathematical power level of students based on grade level. This assessment model could evaluate cognitive, 
psychomotor and affective characteristic aspects, so that the measurement result could represent the progress 
of their mathematical power. The model of mathematical power assessment is made into this following 
scheme as presented in Figure 1. 


Model to Assess Mathematical Power Based on SOLO Taxonomy 


Mathematical Power Assessment 


Cognitive Psychomotor Affective 


Aspect Aspect Aspect 


Test and Non Test Instrument Arrangement 


Written Test Observation Observation on 





(Essay) (Response Quality) Students Activity 





SOLO Taxonomy Mathematical 
Disposition 
Meta Analysis < 


Thinking Construction Model and Level 
of Mathematical Power 


Figure 1. Scheme of Mathematical Power Assessment Model 
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Referring to the scheme of mathematical power assessment model based on the SOLO Taxonomy as 
illustrated in Figure 1, this assessment model which will be implemented in schools should follow the 
following steps: (1) determining the learning objectives, (2) determining the material according to the grade 
level, (3) ) preparing the learning instruments (syllabus and lesson plan) and learning media, (4) preparing the 
research instrument i.e. test instrument (essay writing test) compiled based on mathematical power ability 
and SOLO Taxonomy level and non-test instruments (student activity observation sheets and interview 
guidance books), (5) managing mathematics learning by applying learning models and approaches which can 
improve and develop students’ mathematical skills, e.g group investigation, discovery learning, inquiry, 
problem based learning and other similar models, (6) observing student activities in learning, (7) conducting 
essay writing tests, (8) conducting observation on students’ answer sheet to obtain data of students’ response 
quality (9) conducting in-depth interviews on selected subjects (students) with purposive snowball sampling 
technique, (10) recording, analyzing and describing interview results, (11) conducting a meta-analysis that is 
combining and discussing the research result between quantitative and qualitative data, and (12) drawing 
conclusions so that the level description and students’ mathematical thinking construction can be obtained. 


2. RESEARCH METHOD 

This research used descriptive quantitative approach which was conducted through field 
research [22]. The subjects of this research were 20 eighth graders in Sultan Agung Islamic Junior High 
School 4 Semarang, Indonesia. The instruments developed in this research included test instruments (essay 
items) and non-test instruments (guide book of observation on investigative activities [23] and mathematical 
disposition questionnaires [17]). Quality of test items was reviewed its validity, reliability, difficulty and 
discriminating power. Before tested, the test instruments’ construction were validated by expert team as the 
reviewer. Data collection technique was conducted by test, documentation, and observation. Quantitative 
analysis on the test items was based on the data from the items that had been tested. 

The research procedure was conducted by following stages: (1) field (school) observation; (2) 
determining research samples; (3) arranging research instruments; test instruments (written and essay 
questions) referring to mathematical power ability and SOLO Taxonomy, and non-test instruments 
(observation on students activity) which were compiled based on learning activity instruments; (4) research 
instrument tests i.e. test and non-test instruments; (5) conducting analysis of instrument test result; (6) 
arranging quantitative description; (7) providing conclusions; and (8) compiling research result report. 

The development of mathematical power test instruments was conducted by two phases; arranging 
test instrument and test instrument validation. These phases arethe improvement of ideas obtained from the 
author’s previous research result [17]. The following is a scheme showing relevant activities for each phase 
of developing test instruments as mathematical power measurement on Figure 2. 
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Figure 2. The Phases of Developing Instruments Modified by Beyers [1] 
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Phase 1. Instrument Arrangement: First phase in preparing mathematical power test items was a field 
study by conducting observation and interview on mathematics teachers in schools and reviewing literature 
on mathematical power from various sources. Then, a written test outline was prepared based on preliminary 
data from the teachers and relevant literature review prepared. While arranging and assembling, test 
instruments were re-evaluated based on peer and expert suggestion. Finally, the test was ready to be 
conducted in the field (school). 

Phase 2. Instrument Validation: Instrument analysis technique in the first stage i.e. validation test 
was used to determine the validity level of test items. The formula used to determine the items’ validity was 
Correlation Product Moment with variables [24]. The calculation of correlation coeffecient value was 
consulted to the value of r product moment so it could be determined if the correlation is “significant or not”. 

In second stage, the reliability of test items was calculated by Cronbach's Alpha formula [17]. After 
the coefficient value of r;; was obtained, it was consulted to r product moment value. The instruments were 
said to be reliable if r11 > ftbi. The third stage was to analyze the difficulty level of the items. The item 
instruments were compiled with high difficulty (hard), medium and easy levels with proportionally 
determined percentages and considering SOLO taxonomy level criteria. 

In fourth stage, it was to determine the discriminating power of the items. Discriminating 
powermeansan ability of a question/item to distinguish between high-ability of students and low-ability of 
students. To determine the discriminating power of the items, t test was used [17]. Value of t. obtained 


count 


was then consulted to t value. The test instrument was said to be significant if value of toaa Z fable With 


table ount — “table 
5 % of trust level and dk = (nı — n2). 

Besides, non-test instruments in forms of learning activity observation sheets (investigative activities) 
developed from the author's research [21, 23] (see appendix 2) and questionnaires (see appendix 3) were 
compiled with appropriate indicators [17, 18], and then were tested to students in school. The validity and 
reliability level of observation result data of student activities and the questionnaires were analyzed as steps 


conducted in test instruments development phase [4, 25]. 


3. FINDINGS AND DISCUSSION 

After completely compiled by the authorsbefore used as a test, the instrument test should be 
constructively validated by the expert team consisting of Prof. Dr. Sunandar (reviewer 1), Dr. Rasiman 
(reviewer 2), and Dr. Isnarto (reviewer 3). The reviewers conducted assessment with a scale of 1 to 5. The 
assessment result was recorded on validation sheet represented in x score. The obtained average score (x) 
shows the construction validity test of mathematical power ability implemented in learning. 

Based on the results from expert team’s validation on test instruments, it was obtained that score for 
each validation aspect was more than 4.20 and the total average score was 4.37. The results from reviewers’ 
assessment on test construction validity showed a very good category with a percentage of 87.4%. 
Furthermore, it can be concluded that the test instrument is eligible for the research without any revisions. 

Secondly, the instrument of written test consisting of 4 (four) items had been tested to 20 students. 


The results of the test instrument were recorded and analyzed for each item. The results of the test item 
validity used product moment correlation formula with 20 students as respondents with 5% significance 
level. Item number 1 has rx value = 0.985, number 2 has rxy value = 0.965, number 3 has rxy value = 0.970, 
and number 4 has rxy value = 0.975. 


An item is said to be valid if it showed rw > Fabie, and if Try S Trable then the item was invalid. The 
Value of Fabie for N=20 was Fabie = 0.444. So 100% of the items are said to be valid, meaning that all four 
items are in line with the criteria and not diverged from the reality. All those four items have valid criteria so 
they could be used as the test instruments. The items could be instantly used or inserted to items file as 
archieve. Calculation of realibility coefficient used Alpha formula with 5% significance level. The result 
showed that the value of Fabie was 0.444, and rı; was 0.974. The Interpretation of realibility coefficient (777) 
was on interval 0.70 < r7; < 0.90 meaning that the instruments has high reliability. 


Third stage testing was test item difficulty. The result of the item analysis on the difficulty level of 
each question is the fourth question has a score between 0.25 to 0.75 with the category "medium" or "quite 
difficult". There were four or 100% items on ‘medium’ or ‘quite difficult’ category. The fourth stage of 
testing is the test of differentiation of test. 

Based on the analysis that has been obtained that, the percentage of items with "satisfactory" 
discriminatory power was 100% . The items with "satisfactory" discriminating power were those which had 
discrimination index had 0.20 to 0.39. The result of this study was reinforced by the opinion of experts 
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saying that to determine the discriminatory powerof items was very important, because one of basic guidance 
to arrange test items was to assume that students in class had different ability [24]. 

This research used four test items. These fouritems represented the types of questions on aspects of 
mathematical power ability: reasoning, mathematical connection, and mathematical communication. Criteria 
of the items used were valid, reliable, having good discriminatory power, and considering the difficulty level 
of each item. The following table presents the matrix of item qualification used to measure mathematical 
power based on SOLO taxonomy. 


Table 1. Data of Item Qualification Test 


No. Mathematical Characteristics in Dificulty Validity Reliability Distinguishing 

Power Ability SOLO Taxonomy Level Power 

1 Mathematical Extended Abstract medium valid reliable satisfactory 
Communication 

2 Reasoning Relational medium valid reliable satisfactory 

3 Mathematical Realtional Medium valid reliable satisfactory 
Connection 

4 Reasoning Extended Abstract medium valid reliable satisfactory 


Table 1 provides data indicating that the four chosen items are qualified to be used as written test 
instruments to measure student mathematical power. The four items have met the aspects of mathematical 
power, and SOLO Taxonomy based on the grading level with a good discriminatory power, proportional 
difficulty level, and also high validity level and realibility level (see appendix 1). 

The outline for test items which was developed for four written test items has met mathematical power 
ability aspects: reasoning, mathematical connection and mathematical communication, and has met 
characteristics in SOLO Taxonomy based on VIII grade and intended learning outcomes (ILO) levels: 
relational and extended abstract level [16]. Besides, the result of mathematical power ability test indicated 
that the highest score was 85, the lowest score was 50, and the average score was 67. These result indicated a 
fairly good category. The validity test was performed by observing construction process of arrangement 
mathematical power research instruments using outline (see appendix 1). In the outline, all aspects were 
included.They are 3 aspects of mathematical power ability. 

Based on the analysis of observation sheets of activity investigation (appendix 2), all indicators of 
observation sheets in 20 items are valid andreliable. The reliability test showed the value of r = 0.912. That 
value was above /iable value on 0.444 with 0.05 significance level and sample n = 20. Meanwhile, the analysis 
on mathemathical disposition quistionnaires (appendix 3) pointed out information that all indicators of 
mathematical disposition in 20 items were valid and reliable. The reliability test on the questionnaires 
showed the value of r = 0.770. That value was above r (table) value on 0.444 with 0.05 significance level 
and sample n = 20. Thus, the reliability aspect of the instruments had completed. Then, a better reliability and 
validity test was performed in the next stage, on much greater samples and should was socialized. 


4. CONCLUSION 

Based on the testinthe field andthe analysis of instrument qualification, it could be concluded that: 
(1) the steps in the model (scheme) to assess mathematical power could be performed well without any 
obstacles, so it could be said that this assessment scheme model could be used as a tool to assess 
mathematical power, (2) test and non-test instruments are valid and reliable, and (3) prototypes of test and 
non-test instruments could be used as a measurement in mathematical power assessment. Result of this study 
could be instantly used by teachers as well as students at school, or as a reference in the assessment in order 
to develop mathematical power.Then, if the model (scheme) will be used as a measurement, the author 
suggest as follows: (a) Assessment on student mathematical power should be performed thoroughly,including 
cognitive, psychomotor, and affective aspects; (b) Test items should be in forms of non-routine problem 
solving referring to development of student mathematical power and considering characteristic competence 
in SOLO Taxonomy based on grade level; (c) Teacher should be optimal as a facilitator in the learning; (d) 
Usage of instruments in investigative activity observation should be performed repeatedly in order to obtain 
accurate and representative data. 
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Appendix 1: 
Table 7. Outline of essay (written) test items 











Mathematical Power Intended Learning Outcomes Examples of Test Items Item 
Abilit Number 
SOLO 4 Level: In a village, there is a water reservoir with 6 m length, 1 
Mathematical Extended Abstract 5 m width and 4 m height. The reservoir is full of clean 
Communication Students have the ability to water and will be distributed to people's homes. There 
think conceptually, and can are 40 homes. Each house gets 500 liters of water per 
generalize in a new area. day. How many times do the PDAM refill the water in 
order to meet the needs of 40 homes within 1 month? 
SOLO 3 Level: Abdullah wants to make a tent of fabric with the model 2 
Reasoning Relational and size as shown in the picture below. 
At this level, students 
understand how to construct the 
whole and the relationship 
between structures that 
construct the whole. 
0.5m 
6m 10 m 
What is the area of fabric in square meters (m°) 
required by Abdullah to make 1 (one) tent along with 
its base? 
SOLO 3 Level: The roof of a traditional house is pyramid-shaped. The 3 
Relational side of the pyramid is rectangular with 18 m length x 
Mathematical At this level, students 10m width and 12 m heightIn order to prevent leaks, 
Connection understand how to construct the a large plastic cover should be installed on rooftiles. 
whole and the relationship Each 1 m? plastic cover is Rp 5,250.00. What price 
between structures that should be paid to buy that plastic cover for the roof? 
construct the whole. 
SOLO 4 Level: Below is a right-angle triangular prism ABC.DEF. 4 
Reasoning Extended Abstract D = 
Students have the ability to k " 
think conceptually, and can 
generalize in a new area. 
A B 





The prism contains water as CH height. Comparison of 
CH : HF length = 3 : 1. The base ABC has a right angle 
on C. Length of AC = 8 cm while AB = 10 cm, and 
height of AD = 16 cm. 

If the water inside the prism is moved to a cuboid with 
16 cm base length, 6 cm base width and 8 cm on its 
height, what is the length of water inside the cuboid? 
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Appendix 2: 
Table 8. Checklist sheet of observation on investigative activities 


Investigation 
problems from teachers) 


Students are excited to join with their group to learn the topics 


chosen based on their interest 


Planning Students prepare notebooks, 
equipments 
Students read the lesson/learning resources and work instructions 


in the worksheet 

Investigation Students take note of important points in the investigation 
Students conduct discussions and question and answer section in 
the investigation group 


Students help each other understand the lesson 
Students are excited and persistent to do investigation 


Students can illustrateproblem of investigation form of figure, 
sketch, tables, graphs etc. 


Students deliver opinions and ideas for presentation materials 


12 Students actively in discussions make decisions about the 
subjects to be presented 


Students prepare reports to be presented well 


Presenting 14 Students present topic presentation that was studied before in a 
coherent and systematic wa 





Students perform a presentation of group investigation result 
with confidence 

Students perform positive attitude and pay attention to the 
presentation from other groups 


Evaluating 17 Students compile reports which are easily understood by others 
Students record feedback and suggestions from teachers, or 
friends of other groups 
words 

20 


Students provide suggestions and evaluation on the learning 
topic and on the effectiveness of experiences in investigative 
activities. 
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Appendix 3: 
Table 9. Outline of Questionnaire for Mathematical Disposition 
Modes of Mental Subcategory of Statement of Questionnaire Number 
Functioning dispositional function 
Dispositional Connections I believe that, in mathematics, there is a relationship between topics or 1 
cognitive lessons each other 
function I am not able to relate ideas or topics in and across mathematics 11 
Argumentation I believe that, by correcting or re-examining my work, the result will be 2 
more satisfying 
I can give a good reason for the result of my task 12 
Afektif Nature of Mathematics I do not believe that mathematics consists of systematic concepts or 3 
disposisional rules or procedures 
I am currently solving mathematical problems based on mathematics 13 
concepts or procedures 
Usefulness I am convinced that the mathematics I learned is useful for improving 4 
school achievement and succeeding future goals 
I use mathematics knowledge in everyday problem solving 14 
I believe that my mathematics task assessment resultis appropriate and 5 
Worth Whileness qualified for me 
I consider my mathematics achievement to be a valuable experience 15 
Sensibleness I believe that mathematics consists of ideas that can be interpreted 6 
I communicate mathematical ideas through symbols, tables, graphs, or 18 
diagrams to explain mathematical problems 
Mathematics Self- I feel confident that I can develop various ways to solve mathematics 7 
Concept problems 
I do not care if I do not understand or not good at mathematics 20 
Attitude I consider that mathematics lessons are not fun to learn 8 
I feel that mathematicsdoes not fulfil my needs 16 
Math Anxiety I feel anxious when I will have a test or mathematicsexam 9 
I am shy (low self confidence) if my mathematics score is not good 19 
Dispositional Effort/Persistence When I get to have a difficult mathematical problem, I feel challenged to 10 
conative function work hard to find a solution 
I seek for and read lessons from other sources to extend my knowledge 17 


and understanding on mathematics 
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